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This paper examines improving learning outcomes 
through redesigning information landscapes. The concept of 
information landscapes has been a constant theme in the development 
of interactive multimedia packages. For the user interface to this 
information to be effective and efficient, consideration must be 
given to the cognitive load placed on the user. The issues of 
redesigning learning outcomes are examined in the context of the 
development of a CD-ROM based interactive multimedia package, 
"Investigating Lake Iluka M (ILI) , ILI is based around an ecology 
simulation and employs a number of different interface metaphors in 
presenting the materials to the user. The problem-solving nature of 
ILI lends itself to metacogni t ive support through the following 
features: cognitive self-management; provision of prompts; and 
experience of experts. The paper reviews evaluation of the learning 
outcomes from the initial use of the package. (Author/JLB) 
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Abstract 



The concept of information landscapes has been a constant theme in 
the development of interactive multimedia packages. For the user 
interface to this information to be effective and efficient 
consideration must be given to the cognitive load placed on the user. 
Improvement in learning outcomes can be supported by allowing 
students to focus on metacognitive processes as a component of 
performance support. The renewed interest in student-centred 
learning environments and recent constructivist approaches to 
learning place responsibility for learning firmly on the shoulders of 
the student, but this responsibility can only be taken on when 
appropriate support is available and the necessary skills can be 
developed with students. This paper examines these issues in the 
context of the development of a CD-ROM based interactive 
multimedia package, Investigating Lake Iluka', and reviews 
evaluation of the learning outcomes from initial use of the package. 



Over the last decade there has been a significant shift in emphasis in curricula generally. 
Learning basic facts and definitions from textbooks has become less important than the 
application of knowledge in daily life and the development of higher order thinking skills 
such as problem-solving, critical thinking and decision making. In many countries this shift 
has been developing in parallel with national programs which are emphasising a move 
toward a more literate populus. The quality of the learning outcomis of a nation's education 
and training systems play a central role in determining the future levels of economic and 
social development. Many in-house industry training programs have realised that the focus 
should be on effective performance and problem solving rather the ability to remember facts 
and repeat theory without real understanding about its applicability 

Recent curriculum documents in many western countries emphasise the skills of 
investigation, reflection and analysis to generate or refine knowledge. The appeal of 
cognitive process training to support this development is obvious, and it seems far more 
efficient to provide the student with general-purpose problem solving than instruction on 
specific solutions to specific problems. Metacognitive support provides a key to efficient 
higher order learning. 

There has been considerable controversy, which will no doubt continue to simmer, over the 
clarification of constructivism as opposed to subjectivism. The constructionists argue that 
learning outcomes depend on: 

• the learning environment. 

• the prior knowledge of the learner. 

• the learner's view of the purpose of the task. 

• the motivation of the learner. 

The process of learning involves the construction of meanings by the learner from what is 
said or demonstrated or experienced. The role of the teacher is one of facilitating the 
development of understanding by selecting appropriate experiences and then allowing 
students to reflect on these experiences. 

To the learner, the constructivist learning experience may not look welcoming. It may seem 
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daunting and complex to those who feel ill-prepared for such creative freedom and choice of 
direction. Often constructivism learning situations suddenly throw students on their own 
management resources and many fend poorly in the high cognitive complexity of the learning 
environment. Cognitive support tools and the explicit acknowledgment of the double agenda 
of metacognitive self- management and learning can help. The scaffolding and coaching of the 
cognitive apprenticeship model offer another solution. 



Multimedia Design in a Contructivist Framework 

A number of multimedia design models have been developed which illustrate the 
combination of complex learning environments and which also give students their own real 
control over their learning environment. Our model (Figure 1) is based on a more organic and 
iterative approach than traditional instructional systems design. Phase one takes the basic 
information derived from a needs assessment and converts it into a description of the Project 
space — the information which is to be included in the materials, how it is structured, what 
the target audience understands about the information and how it might be structured for 
the audience. A possible structuring device might a concept map of the ideas and links that 
are to be included in the project. 

The second phase reviews the basic description and seeks to link the elements through an 
appropriate instructional or presentation strategy. It also seeks to identify metaphors which 
help both the design team and the final presentation of the information structure. The 
outcome of the second phase would be a formal description such as a design brief. The detail 
would enable the reader to understand the underlying knowledge structures and the ways it 
is proposed to link them conceptually and intuitively. 
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Interaction Analysis 
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Figure I: The design process used for this interactive multimedia package. 
The third phase is a third pass at the same material, this time with the express goal of 
linking the design ideas into a potential interaction structure. One output of this phase 
would bo an interactive mock-up of the interactive materials using such tools as HyperCard 
or Toolbook to illustrate not only static display of information but also the graphical and 
visual metaphors used to create understandable links. The information included in this 
prototype may include visual, motion, static graphics, sound and data landscapes as 
appropriate to the concept under development. 

Each interaction consists of a node point which forms the basis of the interaction, a set of 
options which provide links to other nodes or additional information attached to the current 
node. One of the links must relate to earlier travelled or preferred paths through the 
materials, and each choice must inform the user about what is likely to occur as a result of a 
choice. These can translate into the traditional concept of results (correct or incorrect) or 
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information feedback choice, but should also include simple feedback elements such as 
confirmation of choice (feedback that a button has been selected) or performance support 
enhancement such as suggested hints, or revision of the underlying concept/principle which 
might be employed to make the choice. Depending on the instructional strategy chosen 
another element might include the concept of duration, either time or the limit of options 
based upon previous choices or paths taken. What constitutes each of these functions and 
what they create in terms of cognitive skill development for the user are determined by their 
physical manifestation in teims of navigation options. 

The complex integration now possible with a variety of hardware and software combinations 
raises problems for the user in that multiple paths are possible to the same or different end- 
points. Learners are faced with the need to understand what learning possibilities might be 
available from where they are in a multimedia learning environment. When a student can 
branch down multiple paths and rapidly change the direction and focus of the learning 
sequence, there is possible interference with effective learning through the inappropriate 
application of information by the learner to their internal schemas. Other concerns include 
disorientation with location, cognitive overload when following several trails or trying to 
remain oriented on their goal, flagging commitment and a poor presentation rhetoric or 
metaphor. 

The potential of both the technology and learning strategies to incorporate the recent 
initiatives in science education have lead to the development, production and evaluation of a 
particular interactive multimedia CD-ROM based package called Investigating Lake Iluka. 
The package has been designed to facilitate access to the information landscape through the 
learner's choices by: 

• supplying accessible and useable tools to allow access to the scope of supporting 
interactive multimedia resources ( eg. video and graphic representations of concepts) 

* providing an adaptive navigation system and coherent information metaphor which 
requires little or no explanation. 

Investigating Lake Iluka has been based around an ecology simulation and employs a 
number of different interface metaphors in presenting the materials to the user. The package 
is based on the concept of an information landscape that incorporates the biological, chemical 
and physical components of a range of ecosystems that make up a coastal lake environment. 
The user is given some problem solving strategies to investigate this information in a variety 
of ways using the range of physical tools provided. They can collect biological, physical or 
chemical data as well as media information and 'construct' their own understanding of the 
basic ecology concepts embedded in the package. This facility has the potential to increase 
student understanding and control of their learning through control of their learning 
environment. Inquiry and problem-solving techniques have been embedded in the package 
through case studies of ecological scenarios presented to the user via media reports of 
problems posed directly to the user. Each scenario can be investigated using the package 
tools. It is expected that users will develop a broad array of scientific investigation skills 
using this realistic simulation. One of the unique features of this package is the facility for 
users to generate their own customised report which can be refined and presented 
independently of the package. 

Inquiry and problem-solving techniques have been embedded in the package through case 
studies of ecological scenarios presented to the user via media reports of problems posed 
directly to the user. Each scenario can be investigated using the package tools. It is expected 
that users will develop a broad array of scientific investigation skills using this realistic 
simulation. One of the unique features of this package is the facility for users to generate 
their own customised report which can be refined and presented independently of the 
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Figure 2: Investigating Lake Iluka where the user is presented with . . <blem and a set of 
suggestions to help them solve a problem not simply choose an answer to prepared multiple 
choice questions. (Program from Interactive Multimedia Pty Ltd, Old Parliament Hourse, 
Canberra) 



Metacognitive Support in Investigating Lake Iluka 

Paris & Winograd (1990) have defined metacognition as knowledge about cognitive states 
and abilities that can be shared among individuals, including the affective and motivational 
aspects of thinking. Cognitive strategies can be addressed directly in the structure of an 
information landscape. The affective and motivational aspects of metacognition are 
embedded in the interface. 

The problem solving nature of Investigating Lake Iluka lends itself to metacognitive support 
through a number of means: 

• cognitive self-management: Students should form good plans, use a variety of strategies 
and monitor and revise ongoing performance. The notebook within Investigating Lake 
Iluka allows the student to collect and manage information from a variety of sources. 
Transcripts are not provided for video and audio material as this is not consistent with 
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how students would process material in these media outside the package. They must 
organise and edit what is potentially an overload of information, within the broad 
constraints of an open-ended problem. 

The role of the teacher is vital to the editing process at this stage. Students need to be 
encouraged to be critical in appraising the relevance and credibility of material. They 
also need to be guided through the development of a report on the problem posed. 

• provision of prompts: Hints accompany the problems posedwithin each ecosystem. 
Students choose to access these hints, which guide them to explore various areas of the 
landscape or measure certain physical, chemical or biological characteristics with the 
toolkits provided. 

The ro]p of the teacher is to ensure students record data collected in a systematic and 
scientific way. 

• experience of experts: the reference book and a number of media reports give the student 
the chance to gain from the experience of those who specialise in ecology. 

The role of the teacher is to help students evaluate issues of hidden agenda, conflict 
among experts, alternate sources of information and timing of information release. 

You need specific and accessible knowledge to solve problems. An information landscape such 
as Investigating Lake Iluka provides the knowledge base and the knowledge schemas of 
experts in association with a mechanism for the student to collect, analyse, assimilate and 
synthesise responses to problems. The ability to see a bigger picture is facilitated by rapid 
infc/mation access and retrieval. 

Embedded content independent strategies are general learning strategies incoporated within 
available content. They support local learning but emphasise strategy transfer as well 
(Osman & Hannafm, 1992) . A well structured information landscape will provide a 
template for a range of content. The strategies used in Investigating Lake Iluka could well 
apply to other tools and other problems. 



Evaluation Strategy 

The evaluation involved three main approaches. Expert review of the package, one-on-one 
testing of the prototype materials via video observation and interviews and in-depth case 
studies, including the verification of the methods for data analysis of complex multi-path 
data This type of data collection varies from subject to subject and requires the development 
of special techniques for its analysis and interpretation. Examination of the contents of the 
incorporated notebook provided some indication of the pattern of student use of both physical 
and metacognitive tools in the package. This formative aspect of the evaluation was be used 
to guide decisions on debugging and enhancing the package. A further evaluation strategy 
was employed with classroom groups of students, who were set learning tasks individually in 
the multi-media environment, and tracking data collected for analysis. 

Analysis of the collected data provides the group with an interesting opportunity for the 
continued development of techniques to extract the maximum amount of information for 
feedback to developers and those who commission interactive materials. Simple statistics 
such as how many users have used the system and the lengths of time they used it are 
relatively simple to extract, but more sophisticated analyses of how a particular interactive 
package is actually being used (what sections are being used, to what extent, what is not 
being accessed, where use:s exit the system, by what method, and so on) were examined and 
the results used to provide feedback on the future specifications for new versions and added 
features. 
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Expert Review 

A naive expert tester had an educational background in Mathematics and Computers. He 
was asked to review the program and to verbalise his actions and thoughts. He had not been 
introduced to the program before. His commentary was observed and recorded, and 
comments and observations were added later. 

The naive expert's responses were about two main issues: interface conventions and the 
conceptual structure and functionality of the package. 

Interface Conventions 



• Text transfer to the Notebook: he wanted to cut and paste. He tried to "Drag" across the 
text as a method of 'select air, and felt that a click was not a standard method to cut and 
paste. "If I click on there (the text) I get the whole the lot. I can't just pull a sentence off". 
He found the process of editing a little confusing, but realised this was necessary 
otherwise text would be corrupted by the novice user. 

• The recording of measurements in the Notebook: he was confused as to what the pencil was 

used for. He felt once the hand symbol moved into his notes it should automatically turn 
into an "I" beam so he could write. The naive Expert wanted to use the pencil to activate 
this. "I was looking for a tool to turn the hand curser to an "I" bar. 

Conceptual Structure and Functionality 

Early in the testing process, he realised that he needed the help screens to fully understand 
how to use the program. After the introduction finished he worked through all of the Help 
screens. "I now feel that I have an understanding of that". 

Within an ecosystem, the naive Expert selected a problem and used the Notebook to review 
the question and hints. He kept losing the question which was not at the top of the notebook ( 
he had other data in there), as the scroll ba^ jumped back to the top each time the notebook 
was opened and closed. This was subsequently rectified by putting the heading "PROBLEM" 
and a blank line before the problem in the notebook, so it was more easily located. 

He found measurement from the image of the ecosystem a wonderful idea, and set himself a 
problem of investigating wind speed in different areas. It took some trial and error getting 
measurements into the Notebook as the pencil was turned off and also hidden by toolkit. The 
naive Expert expressed the desire to establish a transect for measurements to make a set of 
multiple readings. 

Within the media room, the naive Expert noted the video titles didn't give an idea of content 
and expressed a desire to see these indexed. He opened the notebook while a video was 
playing and began typing. 'That is really good, that you can play the video and enter data." 
Having used the program once, the naive Expert feli confident about its use and moved 
through the media room easily on his second visit while solving a different scenario problem 
He was still confused about the pencil, but with trial and error was able to record 
measurements. 

The naive Expert was impressed with the full screen presentation of downloaded notebook 
contents for editing in a word processor. The main outcome of this evaluation can be 
summarized as a series of guides to faciltators: 

• Work through the Help screens early in the introduction of the program to students. 
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• The Teacher's manual should emphasise that editing capacity is only on the blue pages of 

the Notebook and is activated by clicking within the notes. 

• There should be a demonstration early in the presentation of the program so students 

know to turn the pencil on to begin recording measurements. 

User Evaluation (Teachers and Students) 

Evaluation of the package by teachers and students in the early stages of completion resulted 
in a number of insights into the design features of the application and also helped to focus 
the package objectives. The package was introduced to two classes of year 11 students 
studying the topic of ecology. Extensive observations were made of the student use of the 
package, as well as their response to the percieved outcomes of its use. 

The information landscape structure of the program relies heavily on teachers understanding 
of the processes that can be practised and the nature of the problems posed in the problem 
section of the the program. The performance support tools, such as the notebook, are an 
integral part of the investigative design, but they do rely on good direction by teachers. As 
an example of this issue one of the inbuilt problems in the Mangrove Ecosystem was posed by 
one of the teachers in these initial trials — "What types of animals are adapted to this 
environment?" 

After setting up small groups, demonstration of the navigation process and a brief practice 
with the use of the note book, the class was given the opportunity to use the package over a 
number of class sessions. The expectation from the teacher was that students would read or 
collate the relevant information from the package. They would analyse the information and 
present a synthesised statement of the key characteristics of the animals of the mangrove 
ecosystem. The students had no difficulty collecting the information via the notebook cut and 
paste facility, but for the majority of groups, the teacher recieved a complete printout of the 
"Animal and Plants Book" information for every animal that lived in the mangroves. Thus 
they have not attempted to analyse the pertinent information and remove sections which did 
not answer the question. Two groups did attempt some synthesis of the information, but 
again tended to include rather than exclude irrelevant information. 

Interviews with the students and teachers revealed that they did have skills to synthesize 
such data, and had previously demonstrated these skills, but the students admitted that the 
power of being able to extract, in electronic form, every bit of information on individual 
animals from the package compared with the usual practice for such tasks of having to type 
in their response, or write it out by hand proved to be too attractive not to include all that 
could be collected. Even after a class discussion of the type of report the teacher was 
expecting the importance of synthesising information to produce a concise answer, the final 
reports by students were much longer than the teacher would normally received, if written 
resources were supplied to the students. After using the complete package, the skills 
required together with the necessity to edit and tighten reports was seen as an important 
learning point for students. It was seen as a new skill which up to the availablity of this 
technology had not been an issue. 

Observation of the use of the navigation facilities indicated that the students had little 
difficulty in finding information in the package that they sought. The "help" facilities were 
only used occasionally. It is important to note that all the help was provided in three ways. 
Help on how the package worked and how to use the measuring tools was provided by 
animated movies of the screens with accompaning explanation. Help on how the package 
was organised conceptually was provided through a stack map which was allways available 
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and also allow navigational "jumping" between sections, the third form of help was provided 
through the use of hints and suggestions about where to look to solutions and what might be 
important concepts, this last form was highly context specific and provided specific ideas 
related to the learning task. 



Other Teacher Evaluations 

Information was collected on teachers response to the use of th package with their class and 
also through use of the package in workshops at local conferences. A number of key issues 
arose from this trialling. Some of the recommendations were incorporated in the release 
version, and others will be incorporated in future developments of the package. Most of the 
key issues raised by teachers were not to do with the internal workings of the package, but to 
do with the teaching process and organisation of their classes. 



Many schools do not have large numbers of machines with CD-ROM facilities. Teachers 
were concerned that, even though they believed that the package would strongly support the 
teaching process, lack of technology would reduce the impact, and in some schools mean that 
teachers would not use the package. A strategy to maximise the availability of the program 
to students was adopted. Two versions of the program were placed on the CD-ROM, one 
which relied on the video segments on the CD-ROM disc, and one which did not use the video 
materials and could be copied to hard disc. Each package then represented a licence to use 
twenty copies of the hard disc version and one copy of the CD-ROM with the video segments. 
The other key issue raised by teachers was related to appropriate teaching and learning 
strategies. Many teachers wanted guidance on how to organise cooperative group learning, 
independent research and guided inquiry lessons. It was considered that this teacher support 
could not be incorporated into the package directly, but was added as part of the printed 
materials distributed with the package. There was also considerable interest expressed for 
ideas on how to integrate field trips into use of the package. Ideas on use of the package 
before visiting a field site and on following up the field trip were considered to be important 
to add to the support materials. 

The teachers involved in the initial class testing of the program were particularly excited 
about their students use of the notebook concept and were surprised at the ease with which 
their students used this inbuilt tool. However, they did find that students took notes and 
saved their notes effectively at the end of each session, but had difficulty determining what 
data they had previously collected when they started a new session. Teachers proposed that 
previously saved notes should be able to be read back into the package so that students could 
more easily continue their investigation. This feature has not been incorporated into the 
current version. 

There was also a general consensus amongst the teachers that the package had a much wider 
application than ecology education. Many teachers envisaged application to geography and 
especially English, in the form of media studies. One interesting aspect of the evaluation 
involved discussion of student access to the video and audio scripts. The design team had 
intended to incorporate this feature, but teachers disagreed. They proposed that it was 
necessary for students to develop skills in summarise information from such media sources 
and if the students had access to a textual form, this would negate development of an 
essential skill for students. 

Teachers evaluating the package were particularly supportive of a number of features of the 
package that they saw stongly supporting the teaching and learning process. The features of 
the package noted in this context were:- 
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• the level of interactivity 

• the measuring tools 

• the notebook facility 

• the video and audio resources. 

Their moin concern was that teaching support ideas should be available with the package so 
that they could use the package to the maximum advantage. 

Future Developments 

On the basis of this inital work we have been encouraged that the package does in fact 
require the user to take control of their learning. The metacognitive supports do work well to 
provide a structure and support for problem solving. The main drawback has been the 
reluctance of the students to edit out redundant or unessary text in their reports. As a result 
it is suggested that future versions of the software incorporate: 

• Prompts or advice on report format either though expert opinion or structure. 

• Expert scientists, how they would approach the solution to the problem (Guides) (Hints 
mainly focussed on where to look in the package.) 

• Report generation incorporating other media. 
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